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Introduction. The paper considers improving the management 
quality of an educational organization through automating the 
personalized data collection, storage and ranking. The authors 
have developed a module of the university corporate system, an 
ePortfolio of students, which implements a multicriteria optimi- 
zation method for calculating the students’ rating on the basis of 
the collected and stored data. 

The study object is multicriteria ranking methods. The subject 
of the study is the automated calculation of student’s rating 
within the electronic portfolio. The study objective is to develop 
tools for collecting, storing and processing data on individual 
achievements of students and the implementation of the math- 
ematical method of multicriteria optimization for ranking stu- 
dents on the basis of the portfolio data. Practical implications 
include development of a tool for an effective management of 
the educational process. 

Materials and Methods. A prototype module of the university 
corporate system “ePortfolio” on the platform of 1C: Enterprise 
8.3 is presented. To rank students, a special case of the alterna- 
tive ranking method is implemented in the block of the data 
analysis module — pairwise comparison in order of their relative 
importance. At that, a unified scale of relations was used. 
Research Results. The authors described the scheme of the 
information system (IS) operation “ePortfolio”, presented a 
process interaction pattern for the portfolio formation, as well 
as a diagram of business processes under calculating an indi- 
vidual rating. A fragment of the sample is shown on which the 
performance of the multicriteria optimization block has been 
tested. The criteria of the calculations are described, as well as 


the rule of screening alternatives for solving for the optimum. 


” The research is done within the frame of the independent R&D. 


Beedenue. Padota NocBaAllleHa MOBbIMMIeCHHIO Ka4uecTBa yiIpaB- 
JICHHA OOPa3z0BaTeIbHOU OpraHu3allue 3a CUeT ABTOMATH3A- 
IMM MepCOHHMUIMpOBaHHOrO cOopa, xpaHeHua UW paHxKupo- 
BaHHA JJaHHbIxX. ABTOPpaMU pa3paOoTaH MOJIyJIb KOpMoOpaTuB- 
HOM CHCTeMbI By3a — IICKTPOHHOe MOPTPOMMO CTYACHTOB, B 
KOTOPOM peasIH30BaH MeTOJ| MHOTOKpHTepHasIbHOH ONTHMH- 
3allM JIA pacueta pelituHra OOyyarollluxca Ha OCHOBE CO- 
OpaHHbIX WH XpaHAIMXxcA aHHbIxX. OObeKTOM UCccIeOBaHHA 
ABJIAFOTCA Me€TOJIbI MHOFOKPHTephasIbHOrO paHoKUpOBaHHA. 
IIpeaqMeTom UcciIeqOBaHHA ABIIACTCA ABTOMATH3HpOBaHHBbI 
pacueT peHTHHTa CTyeHTa B paMKax 9JIEKTPOHHOFO TOpTo- 
mmo. Llenbro ucciIeqOBaHuA ABJIAeCTCA pa3spa0oTKa MHCTPyMeH- 
TapusA Wid COopa, XpaHeHuaA U OOpaOOTKH AaHHbIxX 00 HHIMBU- 
WYasIbHBIX JIOCTHXKCHUAX CTYICHTOB HU peasM3allua MaTeMaTH- 
YeCKOrO MeTOJa MHOFOKpHTepvalbHOW ONTHMH3alMu [WA 
paHKUpoBaHHA OOy4aFOIIMXxcA Ha OCHOBe JaHHbIX MOpTo- 
mmo. IIpaxtuueckad 3Ha4YMMOCTb UCCeOBaHHA — Ipeyocta- 
BUTb MHCTPyMeHT JIA 9PeKTHBHOTO yipaBlIeHHA yYeOHbIM 
IIPOI[eCCoM. 

Memoo u uxncmpymenmapuu. IIpenctapiien npoToTHm MOA 
KOpMopaTHBHOM CHCTeMbI By3a «OJICEKTPOHHOe TOpTMoHoy 
Ha nyaTdopme «1C: II[peanpuatue 8.3». Ja pawkupoBaHna 
CTYHCHTOB B OOKe MOYIA aHalIv3a JJaHHbIX peasiM30BaH 
YacTHBIM Ciryuaw MeTONa paH)KMpOBaHHA abTepHaTHB — 
lolmapHoe cpaBHeHve MO CTeMeHH UX OTHOCHTEJIBHOM BaxKHO- 
ctu. IIpu 9sTOM HCHOub30BaIach YHHDUUMpOBaHHas IKasia 
OTHOIeHHH. 

Pe3yibmamol ucciedoeanus. ABTOPbI OnMcasM cxemy (yHK- 
I[MOHMpOBaHHA UHDOpMallMOHHOH cucTemMbl (MC) «Ouek- 
TPpOHHOe NOpTosMoO», MpescTaBMJIM CXeMy B3aMMOJeHCTBHA 
IIpoweccoB 10 PopMupoBaHHO TopTdosMo, a TakxKe CXeMy 
OuM3HEC-IIpOeccoB IIpH pacueTe HHAMBUAyabHOrO peiTuHra. 
Iloxa3aH dbparMeHT BbIOOpKH, Ha KOTOPOHM MpOBOAMIACb Mpo- 
Bepka paOoTOcHOcoOHOcTH OIOKa MHOTOKPHTepHasIbHOH OTI- 
THMU3alIMn. OnvcaHbl KputTepuu, MpuHuMalomlMe yyacTue B 
pacuetax, a TakoKe NpaBusIO OTOOPa albTepHaTHB JIA HaxXO%K- 
J[CHUA ONTHMaJIbHOrO pellleHna. 





™ E-mail: ponomarevala@bk.ru, ox-rom@yandex.ru, zabolotnikovavs@yandex.ru, ponomarevala@bk.ru 


weak 


PaOota BBINOJHeHa B paMKax HHuIMaTHBHOH HUMP. 


Ponomareva L. A., et al. Automation of multicriteria ranking of students using ePortfolio 


IT onomapeé6a JI. Acu Op. Aemomamu3zauua NPpOuecca MHOZLOKPUM CPpualbHOcO PAHICUPOCAHUA cmydenmoe6 C NOMOULbI0 DK MPOHHOcO nopm«osuo 





Discussion and Conclusion. The paper presents the results of 
the IS block operation on ranking a list of students. The calcu- 
lation results coincide with the practical data. 
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Introduction. Assessment of the educational process effectiveness is one of the most essential characteristics 
of the university's activity when calculating its rating [1]. To improve the competitiveness of an educational organiza- 
tion (EO) in the market of educational services, effective monitoring and objective assessment of the educational pro- 
cess results for each student are required [2]. The data is often stored separately or in general lists, which does not pro- 
vide an identifying representation for a future employer. Educational organizations hold lots of events in which a large 
number of students participate. Individual achievements outside the university are also of interest, for example, volun- 
teering, charity, work in summer youth camps, etc. Such information is rarely collected and stored properly. You can 
create a rating for each student based on the collected information. The ranking of students is one of the techniques for 
the effective management of the EO activities [3]. According to the new education standards and under the Federal Law 
273 “On Education in the Russian Federation” [4], automated data collection, storage and processing are required. One 
of the solutions to this problem is an electronic portfolio, which will afford automated ranking of students. 

Research Methods and Tools. The authors have worked out and implemented a module of the corporate sys- 
tem “Electronic Portfolio” on the platform “1C: Enterprise 8.3”. It can also operate as an independent information sys- 
tem [5]. The IS stores information about students' achievements and, on the basis of this data, automatically arranges it 
in descending order. For ranking students, a vector optimization method 1s applied. 

Research Objective is to develop a mathematical model that operates with a set of alternative solutions (alter- 
natives) F(X) (where_X is a certain set of criteria), which will be further arranged in descending order of preference. The 
most preferable solution will be the first, next — the less preferable one, etc. The degree of preference is estimated by the 
value of the calculated rank for each alternative. The rank 1s calculated from the values of the criterion weights that are 
assigned by the decision maker (DM). 

Let the function ’(X;) form an evaluation vector of alternative solutions under the following conditions: 








i=1,m;j= 1,n, 
where m is the number of alternatives in the sample; 7 is the number of criteria involved in determining the student’s 
rank. 
Let F(X) be the estimate of the solution X7 by the first criterion; F',(X;) be evaluation of the solution _X, by the second 
criterion, etc. 
Then the estimation vector for the first solution has the form: 
F(X) = (F(X), F2(X7), Fin(X1)). 
A set of estimates Yp belonging to the admissible region D determined on the basis of data stored in the ePort- 
folio database is identified as a region of a set of alternative estimates: 
D=<C, F, X, G, P> [6], 
where C is the goal, which is to rank the list of students in descending order of the calculated rating based on the ePort- 
folio data; Ff are alternatives, i.e. the names of students to be ordered; X are criteria, i.e., data on the students’ achieve- 
ments stored in the ePortfolio; P are preferences: X, > X,,. These are indicators that correspond to the educational pro- 
cess and are the basis for comparing alternatives and for making decisions; G are restrictions on the region of feasibil- 
ity. 
The alternative /(X;), in which the largest number of the studied criteria take the maximum value, can be 
found from the expression: 


Information technology, computer science, and management 


X; € D; Fi(Xj) € Yo. 
Then the task of ordering (ranking) the alternatives (F;(X;)) on the criterion set D” in descending order can be described 
by the expression: 383 
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Yp = F(D) = {Y |Y=F(X), X € D"}. 

To solve this problem, the authors applied a special case of the method of ranking alternatives — pairwise 
comparison according to the degree of their relative importance using a unified ratio scale [7]. For each DM function 
F(X), the degree of preference up;(X)), j = 1,n was determined by the weight of each criterion. 

Research Results. Automation of the multi-criteria ranking of students occurs in the information system mod- 
ule of their ePortfolio. The operation scheme of the prototype ePortfolio IS based on the developed configuration in 
“1C: Enterprise 8.3” [8] 1s shown in Fig. 1. 
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Fig. 1. ePortfolio operation scheme 


To generate a student portfolio, it 1s provided: 

¢ data input; 

¢ data conversion to electronic format; 

¢ data storage; 

¢ submission of reports including the calculated students’ ratings and ordered lists of departments, faculties, EO. 
The process of generating a portfolio is presented in Fig. 2. 
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Fig. 2. Portfolio generation scheme 
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The database was designed for IS electronic portfolio [9], the transformation model of which is shown in Fig. 3. 
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Student_Code: Uncno (FK) E 
SW_Code: Uncno : 


SW_Name: Crpoxa(30) : 
SW_Type: Crpoxa(25) F 
SW_Date: Mata 

















S| 
Supervisor_fullname: Crpoxa(50) : 
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| : SA_name: Crpoxa(30) : 
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Fig. 3. ePortfolio database model 


The interaction of information processes on the generation of an individual student’s rating based on the in- 
formation stored in the system database is shown in Fig. 4. 
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Fig. 4. Diagram of IS block operation on student’s ranking score formation 


The IS provides for the generation of various reports [10]. For example, a user can display comparative as- 
sessment blocks for educational, sports, social, creative, and scientific achievements. One of the report forms 1s 
presented in Fig. 5. 


+ Student portfolio 
Full name: Aksanenko Svetlana Viadimirovna 
Birthdate: 13.08.1994 0:00:00 
Sex: Female 
Group: MB-151mz 
Address: Moscow, ul. Grodnenskaya, 12, ap. 90 
Telephone: 8-(916)-123-34-44 
Email: akssvet@gmail.com 


Educational achievements: 


Information technology, computer science, and management 


Number Achievement Type Rating 

1 [Mindgames [Competition 

Scientific achievements: 

Number Achievement Organization Type Rating 

1 SAEI HE MCU [Conference 
Social activity: 

Number Achievement Organization Type Rating 

1 Blood donation SAEI HE MCU [Donorship, 
Sports achievements: 

Number Achievement Level Type Place Rating 

1 Institutional Judo ifkaureate [1 
Creative achievements: 

Number Achievement Type Place Rating 

1 Quiz 

2 





Fig. 5. Report in Portfolio form 385 
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The operation of the IS block, which is responsible for ranking students, was checked on a sample of 1000 

people. The following rule was applied to the sampling: 
F(X) > F(X), i= 1,m, 
ig. Fig(X) > Fig(X), 

where Fy (X*) 1s Pareto-optimal solution. 

After possible deletions implemented according to the rule, 250 mutually incomparable alternatives remained 
in the table, and only the best ones — from previously comparable ones. In this way, a sample was formed for further 
studies, a fragment of which is presented in Table 1. 


Table 1 
Sample Fragment for Ranking 
_._..._. {Participation ; Publication | Participation 
Participation| | Union Increased : : ; : : 
No. Full Name . : in sports __ | Scholarship of sci in sci Volunteer| Donor | Charity 
in Olympiad member | scholarship 


competition papers conferences 
- - +. + 





1/Kolomoytseva Galina Yuryevna _| 
2|Martynenko Elena Anatolyevna | x 
3|Yurchenko Lyudmila Mikhaylovna | 
4|Konovodov Yury Nikolaevich 


Xx x x 





x< 
x< 
x< 





=< 
x 
=< 





























x x x 
5|Alieva Antonina Leonidovna | x x | x x | x | x | 
6|Baytsar Aleksandr Pavlovich | x | | | x x 
7|Smirnova Elena Vasilyevna | x | x | x x 


Optimization was carried out according to 13 criteria: participation in scientific conferences, availability of 
scholarships, participation in public organizations, etc. Further, a matrix of criteria was built, where weight was as- 
signed to each criterion in accordance with the preferences of the DM (Table 2). 














Table 2 

Weighting Matrix 
Criteria X1 X2 X3 x4 X5 X6 X7 xX8 x9 X10 X11 X12 X13 Gmean 
Olympiad 
participant X1 1.00 0.14 3.00 0.25 2.00 1.00 1.00 3.00 0.25 2.00 5.00 5.00 3.00 1.2773764 
Participation in 
scholarship 1.00 7.00 1.00 7.00 5.00 7.00 5.00 5.00 7.00 5.00 4.6581847 
Scholarship 0.50 0.20 3.00 0.14 1.00 0.33 0.33 0.33 0.14 0.20 3.00 0.14 1.00 0.434096 
Sci papers 
publication 1.00 0.20 5.00 0.20 3.00 1.00 7.00 5.00 5.00 7.00 7.00 0.33 5.00 2.007042 
sci conf. X7 1.00 0.50 0.33 0.11 3.00 0.14 1.00 0.33 0.20 1.00 3.00 0.33 0.33 0.5143549 
Donorship 0.25 3.00 5.00 0.20 7.00 0.20 5.00 7.00 1.00 9.00 9.00 7.00 7.00 2.4958934 
Charity Kao | oso | 011 | 2.00 | 011 | 5.00 | 014 | 100 | 020 | 011 | 1.00 | 5.00 | 0.20 | 0.20 |0.4581585| 
fompettins fiz | o20 | 034 | x00 | ose | x00 | so | sao | o20 | ore | so | imo | 100 | 017 fosrera 
competitions 0.20 0.14 1.00 0.14 7.00 3.00 3.00 0.20 0.14 5.00 1.00 1.00 0.17 0.6757149 


Member of 
social 
movement 0.33 0.14 1.00 0.20 1.00 0.20 3.00 0.20 0.14 5.00 1.00 0.7389604 


In Table 1, the “x” icon was replaced by the criterion weight. Then we conducted a pairwise comparison of 
students with respect to any criterion, for example, the first one. The comparison was carried out as follows: the highest 
advantage of one student over another is the value of the first criterion weight plus one. The unit is needed so that when 
dividing the unit by the weight of the criterion, a large number is not obtained. This number should be less than the 
maximum weight of the criterion. That is, the weight of the criterion is the maximum value in the table (highest priori- 
ty). In the next steps of data processing, the system performs certain calculations [11]: 


it quantifies the inconsistency of comparisons; 

e it calculates the degree of inaccuracy of comparisons; 
e it defines a common criterion for each alternative; 

e it finds the best solution; 

e it checks the decision validity. 
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Results and Discussion. As a result of the operation of prototype IS ePortfolio block, which is responsible for 
calculating a student’s rating and arranging a list of students provided in descending order, you can get the report pre- 
sented in Table 3. 

Table 3 
Table Fragment with Student Ranking Results 


X1 X2 X3 X4 X5 X6 X7 X X xB Final values 


8 x9 10 X11 x12 
Fl [ 4.69 9.93 0 10.99 2.43 0 2.56 o | 0 0 0 0 4.08 | 5.213001 
F3 0 2.52 0 0 614 ~— oO 3.16 0 0 4.08 | 0.7033417 
0 9.93 o | o | 243 [ 0 0 0 9.42 3.16 1.61 0 o | 3.2877994 
4.69 0 2.52 7.48 2.56 o0 -F o 0 0 4.05 0 4.1809328 
4.69 0 0 0 0 0 614 — oO 3.16 1.61 0 4.08 | 0.9819622 
0 0 
o0 - o 0 0 
0 0 0 0 
ro 








Tl 
i 











in| 
ui 





n 
a 





4.69 9.33 0 0 0 2.1988993 
F10 a Mes 0 1.39 7.48 6.14 1.61 0 4.08 5.9923473 


In the table, Fl — F10 are the names of students; the top row shows the names of the criteria by which the rank 


nn 
© 











of each object was calculated; the second row from the top is the criterion weight; other items are quantitative assess- 
ments of a student for participating in a particular activity. The DM gave highest preference to the criteria that were 
associated with the study (scholarship, increased scholarship, scientific activity, etc.). According to the calculation re- 
sults, the first four places will be taken by students who receive increased scholarships. F10 student has the highest rat- 
ing, he receives an increased scholarship, has scientific publications and is a participant in sports competitions. The IS 
provides for various different achievements of the same type (for example, a sports category, a winner or a participant 
in a competition) to be combined into one cluster; it affords making a general assessment of the cluster (to present as 
one criterion with a certain weight in calculations). 

Conclusion. The authors propose a tool that provides the user with the opportunity to quantify the studied ob- 
jects with a large amount of information about these objects. The electronic portfolio allows maximum consideration of 
various achievements of students not only within the walls of the university, it accommodates the data accessibility for 
the students themselves, and stores a large amount of information about the student. Calculation of individual rating, 
and ranking improve the educational process quality, which as a result will affect the overall rating of the university. 
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